Image capturing apparatus performing development processing on composite image and control method thereof

ABSTRACT

An image capturing apparatus comprises an image capturing unit configured to capture images by an image sensor; an acquisition unit configured to acquire information relating to sensitivities of the image sensor used for capturing the images; a composition unit configured to add a plurality of images for generating a composite image; and a development processing unit configured to perform development processing of data of the composite image generated by the composition unit; wherein a plurality of parameters having different characteristics corresponding to the sensitivities of the image sensor are provided as the parameters used for the development processing; further comprising a control unit configured to perform controlling to compare the sensitivities used for shooting the plurality of images and set the parameter corresponding to the highest sensitivity as the parameter used for the development processing.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to the technique for multiple shooting ofimages.

2. Description of the Related Art

Multiple shooting is a technique to create splendid visual effects thatcannot be achieved from a single shot by composing an image either frommultiple shots of an object or from an image decoded from an image fileand a shot image. As used herein, the processing of adding a piece ofimage data to another is referred to as composition and repeatedshooting of images is referred to as multiple shooting.

For example, Japanese Patent Laid-Open No. 10-271427 describes a methodof adding a shot image and a separate image for composition to generatea composite image. This method comprises converting the image forcomposition based on information obtained during the shooting of theshot image before generating the composite image. Additionally, JapanesePatent Laid-Open No. 2002-300372 describes a method of recordingcomposite image data along with information for restoring thepre-composition image data.

However, if multiple shooting is performed under conditions withdifferent ISO speeds or conditions with different resize rates ordifferent brightness correction gains, images with different amounts ofnoise, dynamic ranges, and resolutions are added to generate a compositeimage. This may results in inadequate parameters for developmentprocessing, thus adversely affecting the appearance of the image.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of theaforementioned problems, and realizes a technique for using optimumparameters for performing suitable image processing so as to obtainimages with low noise even when multiple shooting is performed underdifferent shooting conditions.

In order to solve the aforementioned problems, the present inventionprovides an image capturing apparatus comprising: an image capturingunit configured to capture images by an image sensor; an acquisitionunit configured to acquire information relating to sensitivities of theimage sensor used for capturing the images; a composition unitconfigured to add a plurality of images for generating a compositeimage; and a development processing unit configured to performdevelopment processing of data of the composite image generated by thecomposition unit; wherein a plurality of parameters having differentcharacteristics corresponding to the sensitivities of the image sensorare provided as the parameters used for the development processing;further comprising a control unit configured to perform controlling tocompare the sensitivities used for shooting the plurality of images andset the parameter corresponding to the highest sensitivity as theparameter used for the development processing.

In order to solve the aforementioned problems, the present inventionprovides a control method of an image capturing apparatus which has animage capturing unit for capturing images by an image sensor, anacquisition unit configured to acquire information relating tosensitivities of the image sensor used for capturing the images, acomposition unit configured to add a plurality of images for generatinga composite image, and a development processing unit configured toperform development processing of data of the composite image generatedby the composition unit; the method comprising the steps of: providing aplurality of parameters having different characteristics correspondingto the sensitivities of the image sensor as the parameters used for thedevelopment processing, and performing controlling to compare thesensitivities used for shooting the plurality of images and set theparameter corresponding to the highest sensitivity as the parameter usedfor the development processing.

According to the present invention, images with low noise can beobtained by using optimum parameters for performing suitable imageprocessing even when multiple shooting is performed under differentshooting conditions.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the configuration of an apparatusaccording to an embodiment of the present invention.

FIG. 2 is a flowchart showing a process of multiple shooting and aprocess of generating a composite image according to a first embodiment.

FIG. 3 is a block diagram showing a configuration of a developmentprocessing unit of the first embodiment.

FIG. 4 is a flowchart showing a process of multiple shooting and aprocess of generating a composite image according to a secondembodiment.

FIG. 5 is a block diagram showing a configuration of the developmentprocessing unit of the second embodiment.

FIG. 6 is a flowchart showing a process of multiple shooting and aprocess of generating a composite image according to a third embodiment.

FIGS. 7A-7J illustrate examples of arrays of pixels in developmentprocessing.

FIGS. 8A-8C illustrate examples of filter coefficients used in luminanceand color noise reduction processing.

FIGS. 9A-9B illustrate examples of color gain characteristics withrespect to luminance signals in color gain processing.

FIGS. 10A-10B illustrate examples of input/output characteristics ofimage data in luminance gamma processing and color gamma processing.

FIGS. 11A-11C illustrate examples of image data level characteristics insharpness processing.

FIGS. 11D-11E illustrate examples of gain values of an edge enhancementcomponent.

FIGS. 12A-12B illustrate examples of input/output characteristics ofcolor signals in saturation correction processing.

DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention will be described in detail below.The following embodiments are merely examples for practicing the presentinvention. The embodiments should be properly modified or changeddepending on various conditions and the structure of an apparatus towhich the present invention is applied. The present invention should notbe limited to the following embodiments. Also, parts of the embodimentsto be described later may be properly combined.

The present invention will be described hereinafter as applied to imagecapturing apparatus for shooting moving and/or still images, such asdigital cameras. The present invention, however, can be applied to othermobile electronic instruments, such as smart phones.

First Embodiment

A first embodiment will be described hereinafter with reference to FIGS.1-3. Throughout the specification, the processing of adding images isreferred to as composition and repeated shooting of images is referredto as multiple shooting.

In this embodiment, an object is shot in multiple shots, image data isrecorded in a memory along with the shooting conditions for each shot,and, after image composition, optimum parameters are selected from aplurality of development parameters corresponding to the shootingconditions so as to perform development processing.

Apparatus Configuration

A configuration and operation of a digital camera according to thisembodiment will be described hereinafter with specific reference to FIG.1.

The digital camera of this embodiment has a function of generating acomposite image by adding multiplely shot first image data with secondimage data or, instead of the second image data, third image datadecoded from an image file that has been already recorded. Inparticular, the digital camera adds linear image data that are read outfrom an image sensor 101 and converted to digital signals and thenperforms nonlinear development processing to complete a final image.This maintains the linearity of the images added for composition andrealizes composition processing that linearly corresponds to theexposure values of the shooting. In other words, image composition fromimages shot at the same exposure value will produce image qualityequivalent to that shot at an exposure value of twice the exposure (1EV).

According to a shooting instruction signal 111 from an operation inputunit 110, a CPU 112 outputs an image capture control signal 113 to animage capture control unit 114. The operation input unit 110 iscomprised of a variety of operating members, such as various types ofswitches, buttons, and a touch panel, for receiving various instructionsbased on user operations.

According to the image capture control signal 113 received from the CPU112, the image capture control unit 114 drives the lens, diaphragm, andshutter of the optical system (not shown) and outputs a driving signal115 to the image sensor 101, thereby capturing the object image formedon the imaging surface of the image sensor 101 via the optical system.The image sensor 101 is a single plate image sensor that includes animaging sensor, such as a CCD or CMOS, for converting the object imageinto an electrical signal and has color filters for multiple colors ofred (R), green (G), and blue (B) arranged in a Bayer array.

An analog signal outputted by the image sensor 101 is converted by ananalogue-digital (A/D) conversion unit 102 into a digital signal 103 andrecorded as the first image data at a predetermined address of thememory 104.

According to an image record instruction signal 111 from the operationinput unit 110, the CPU 112 outputs an encoding control signal 121 to anencoding unit 106. According to the encoding control signal 121 receivedfrom the CPU 112, the encoding unit 106 reads out the first image data105 from the memory 104, generates and records an image file 107 of theimage data in a recording medium 108, such as a memory card.

The image data in the image file 107 recorded in the recording medium108 is called RAW data, which is linear data on which nonlineardevelopment processing is yet to be performed (also referred to as “RAWimage data” hereinafter).

A shooting condition acquisition unit 116 acquires from the imagecapture control unit 114 shooting condition information 117, such as ISOspeed (sensitivity), shutter speed, f-stop, focal length, and focusposition, used for shooting the first image data recorded in the memory104, and sends the information to the CPU 112.

Next, the CPU 112 generates second image data to be added with the firstimage data for generating a composite image. If the second image data isgenerated by shooting as in the case of the first image data, the objectimage is captured by following the same procedure as for the first imagedata and recorded at a predetermined address for the second image datain the memory 104. However, if RAW image data is already recorded as animage file in the recording medium 108, the image data may be decodedfrom that image file to obtain second image data.

In response to a file readout instruction signal 111 from the operationinput unit 110, the CPU 112 outputs a readout control signal 126 to therecording medium 108.

According to the readout control signal 126 received from the CPU 112,the decoding unit 161 reads an image file 160 from the recording medium108, decodes the image file 160 back to its original, pre-record RAWimage data 162, and subsequently records the image data at apredetermined address for the second image data in the memory 104.

A shooting condition acquisition unit 163 acquires shooting conditioninformation 164 from the image file 160 that is read by the decodingunit 161 from the recording medium 108 and sends the information to adevelopment control unit 100. The shooting condition informationincludes the ISO speed, shutter speed, f-stop, focal length, and focusposition, etc.

Next, the CPU 112 generates a composite image from a plurality ofmultiplely shot images or decoded image data.

The first image data and the second image data recorded in the memory104, before being added to generate a composite image, is subjected toresizing processing by two-dimensionally spatial linear interpolation ata resizing unit 131 and to digital gain processing at a brightnesscorrection unit 133. If the first and second image data is added togenerate a composite image by using the same number of pixels andmatching the angle of view for both pieces of image data, the number ofpixels of the smaller image data is resized to match the size of thelarger image. Alternatively, to change the brightness of one or both ofthe first and second image data, digital gain is applied to the relevantimage data. In particular, the resizing unit 131 and the brightnesscorrection unit 133 read the first and second image data 130 from thememory 104 and perform resizing and brightness correction according tothe resizing control signal and the brightness correction control signal119 received from the CPU 112.

Furthermore, the CPU 112 outputs a white balance (WB) control signal 120to a WB processing unit 135 according to a WB instruction signal 111from the operation input unit 110. The WB processing unit 135 thencarries out white balance processing on the image data 132 after theresizing and brightness correction according to the WB control signal120 received from the CPU 112.

After undergoing the white balance processing, the first and secondimage data is recorded in their predetermined areas of the memory 104 asthe third and fourth image data 134, respectively. In this way, onceimage processing is completed, the first and second image data issequentially written back to the memory 104 as the third and fourthimage data, respectively.

Next, the CPU 112 outputs a composition control signal 122 to acomposition processing unit 151 according to the WB instruction signal111 from the operation input unit 110. In response to a compositioncontrol signal 122 received from the CPU 112, the composition processingunit 151 reads the third and fourth image data 150 from the memory 104and performs addition of the pixels at the same positions.

A composite image data 152 generated at the composition processing unit151 is subjected to development processing at the development processingunit 141. The development processing unit 141 comprises a circuit thatcan be also used for developing RAW image data 140 obtained by astandard single image shooting as opposed to image data from multipleshooting. As such, the RAW image data 140 and the composite image data152 are both entered into the common input of the development processingunit 141.

The development processing unit 141, according to a development controlsignal 125 for controlling the development parameters received from thedevelopment control unit 100, performs nonlinear development processingon the linear composite image data 152 to enhance the appearance of theimage, subsequently generating and outputting a final image data 142 tothe display device 145. Then, the user can confirm the quality of thefinal image shown on the display device 145.

The final image data 142 is compressed into an image file 144 in theJPEG or other suitable compression format at an encoding unit 143 andrecorded in the recording medium 108.

The development control unit 100 receives the shooting conditioninformation 117, 164, the resizing control signal and the brightnesscorrection control signal 119, the composition control signal 122, andthe development control signal 125 and outputs the development controlsignal 125 for controlling the development processing to the developmentprocessing unit 141 based on the foregoing information and signals.

Multiple Shooting and Image Composition

Multiple shooting and image composition by the digital camera accordingto this embodiment will be described hereinafter with reference to FIG.2.

The process shown in FIG. 2 is implemented by the CPU 112 executing afirmware program recorded in the memory 104.

At s S201-S204 in FIG. 2, the CPU 112 causes the image capture controlunit 114 to control the image sensor 101 to capture an object imageaccording to a shooting instruction signal 111 from the operation inputunit 110, and writes first image data 103 in a predetermined area of thememory 104 after the image data is converted to digital signal at theA/D conversion unit 102. Additionally at S204, the shooting conditionacquisition unit 116 acquires the shooting condition information 117used for shooting the first image from the image capture control unit114 and sends the information to the CPU 112 and the development controlunit 100. The CPU 112 in turn writes the shooting condition informationas well as the first image data 103 in the memory 104.

Next, if receiving another shooting instruction signal 111 from theoperating input unit 110, the CPU 112 writes second image data 103obtained by the second multiple shooting in the memory 104 (S206-S209).Additionally at S209, the shooting condition acquisition unit 116acquires the shooting condition information 117 used for shooting thesecond image from the image capture control unit 114 and sends theinformation to the CPU 112 and the development control unit 100. The CPU112 in turn writes the shooting condition information as well as secondimage data 103 in the memory 104.

If the CPU 112 receives from the operation input unit 110 an instructionto designate an image file recorded in the recording medium 108 ascontaining the image data to be added with the image of the firstshooting to generate a composite image (S205), the CPU 112 reads theimage file 160 recorded in the recording medium 108, causes the decodingunit 161 to decode the file, and writes the decoded second image data162 to the memory 104 (S210-S213). At S213, the shooting conditionacquisition unit 163 acquires the shooting condition information 164relating to the image file read out from the recording medium 108 andsends the information to the development control unit 100. The CPU 112then writes the shooting condition information as well as the secondimage data 162 to the memory 104.

As the next step, the CPU 112 reads the first and second image data 130recorded in the memory 104 (S214) and causes the resizing unit 131 andthe brightness correction unit 133 to perform resizing and brightnesscorrection (S215). The information about the resizing and the brightnesscorrection based on the resizing control signal and the brightnesscorrection control signal 119 is transmitted to the development controlunit 100 (S216).

Then, the CPU 112 causes the WB processing unit 135 to perform whitebalance processing on the first and second image data 132 after theresize and the brightness correction (S217).

At this point, the CPU 112 writes in the memory 104 the third image data134 obtained by processing the first image data and the fourth imagedata 134 obtained by processing the second image data (S218).

Then, if the multiple shooting (or the reading of an image file) iscompleted (S219), the CPU 112 causes the composition processing unit 151to read the third and fourth image data 150 from the memory 104 (S220)and added the data to generate a composite image (S221).

Next, the CPU 112 causes the development control unit 100 to controldevelopment parameters according to the shooting condition information117 and 164 (S223) and output to the development processing unit 141 adevelopment control signal 125 to switch the development parameters.

The development processing unit 141 sets development parametersaccording to the development control signal 125 received from thedevelopment control unit 100 and performs development processing on thecomposite image data 152 (S224).

The processing at S223 and S224 will be described in further detailbelow.

The CPU 112 then causes the encoding unit 143 to generate an image file144 from the final image data 142 obtained by the development processingat S224 (S225) and records the image file 144 in the recording medium108 (S226).

Development Processing

With reference to FIG. 3, the configuration of the developmentprocessing unit 141 and the development processing at S224 of FIG. 2will be described hereinafter.

As shown in FIG. 7A, the composite image data 152 comprising red (R),green (G), and blue (B) pixels arranged in a Bayer array is inputtedinto an interpolation processing unit 302 via an image data input unit301 as in the case of a RAW image data 140 obtained by a single imageshooting.

The interpolation processing unit 302 performs interpolation processingon the composite image data 152 to separate the pixels into image dataof three R, G, and B planes shown in FIGS. 7B-7D and fill in the “0”spaces as shown in FIGS. 7E-7G. This provides each pixel with RGBinformation and these pixels are converted by a matrix conversion unit303 to one-dimensional luminance signal Y (306) and two-dimensionalcolor signals Cr and Cb as shown in FIG. 7H-7J (320). The 3×3 matrixcoefficients used in this processing are given by a matrix conversiontable 304. The matrix conversion table 304 is provided in advance asdesign values that are parameters optimized for the spectralcharacteristics of the RGB color filters of the image sensor 101.

The luminance signal Y (306) separated at the matrix conversion unit 303is subjected to luminance noise reduction processing at a luminancenoise reduction unit 307. In this luminance noise reduction processing,filtering is performed, for example, using a Gaussian filter having thelow pass filter characteristics shown in FIG. 8A. Likewise, color noisereduction processing is performed on the color signals 320 at a colornoise reduction unit 321.

The parameter used at the luminance noise reduction unit 307 is given bya luminance noise reduction table 308 and the parameter used at thecolor noise reduction unit 321 is given by a color noise reduction table322. These parameters are provided in advance as optimized parameters inthe tables 308 and 322 and controlled according to the parameter switchcontrol signal 342 received from the parameter switch control unit 341.

The filter characteristics for the luminance signal may be the same asor different from those for the color signals, and design valuessuitable for the noise characteristics of the camera are used in thisembodiment.

The number of the taps of the filter may be, for example, 3×3 as in FIG.8A, 5×5 as in FIG. 8B, 7×7 as in FIG. 8C, etc., and the noise reductioneffect increases with the number of taps in the filter. Accordingly, afilter with an optimum number of taps is selected from the luminancenoise reduction table 308 and the color noise reduction table 322according to the amount of noise.

Next, in order to reduce the color signals 320 at the color gain unit324 according to the magnitude of the luminance signal 310, gainprocessing is performed on the color signals by applying a negativegain. FIG. 9A shows color gain characteristics according to theembodiment for applying a negative gain to a low luminance portion ofthe luminance signal 310 to make the color noise of the imageinconspicuous and applying a negative gain to a high luminance portionto enhance the appearance of the image and give the image vivid colors.

These color gains are also provided in advance in a color gain table 325as optimized parameters and controlled for color according to theparameter switch control signal 342 received from the parameter switchcontrol unit 341.

Next, nonlinear luminance gamma processing is performed on the luminancesignal 310 at a luminance gamma unit 311, and nonlinear color gammaprocessing is performed on the color signals 320 at a color gamma unit327.

Luminance gamma characteristics are given by conversion tablesrepresenting the correlation between the output luminance with the inputluminance shown in FIGS. 10A and 10B.

Color gamma characteristics are given by converting the color signals Crand Cb with the luminance signal Y to RGB signals, for example, usingthe three-dimensional matrix conversion table for RGB→YCbCr representedby the equation (1) and using the color gamma table of FIG. 10A similarto the luminance gamma. Subsequently, Cr and Cb signals are obtained byperforming matrix conversion using an inverse matrix of the equation (1)as shown in the YCbCr→RGB conversion table represented by the equation(2).

$\begin{matrix}{\begin{bmatrix}Y \\{Cr} \\{Cb}\end{bmatrix} = {\begin{bmatrix}0.2126 & 0.7152 & 0.0722 \\0.5000 & {- 0.4542} & {- 0.0458} \\{- 0.1146} & {- 0.3854} & 0.5000\end{bmatrix}\begin{bmatrix}R \\G \\B\end{bmatrix}}} & (1) \\{\begin{bmatrix}R \\G \\B\end{bmatrix} = {\begin{bmatrix}1.0000 & 1.5748 & 0.0000 \\1.0000 & {- 0.4681} & {- 0.1873} \\1.0000 & 0.0000 & 1.8556\end{bmatrix}\begin{bmatrix}Y \\{Cr} \\{Cb}\end{bmatrix}}} & (2)\end{matrix}$

The gamma characteristics are provided in advance as optimizedparameters in the luminance gamma table 312 and the color gamma table328 and controlled according to the parameter switch control signal 342received from the parameter switch control unit 341.

The gamma table varies to suite the dynamic range of the image sensor101. For example, if the dynamic range is between 0-3500 (12 bits), thecharacteristics 1001 of FIG. 10A are used, whereas if the dynamic rangeis between 0-3000 (12 bits), the characteristics 1002 of FIG. 10B areused.

As the next step, in a sharpness processing unit 314, sharpnessprocessing is performed on the luminance signal Y after the foregoinggamma processing in order to sharpen the image.

Sharpness characteristics shown in FIG. 11C are given to two-dimensionalspace coordinates.

An edge component shown in FIG. 11B is detected by obtaining the secondorder differential coefficient of the spatial variation in the luminancelevel of the luminance signal Y shown in FIG. 11A. If the edge componentis multiplied by a gain Gs and added to the original signal with itssign inverted, sharpness effect may be given to the image.

In this embodiment, a threshold th is set with respect to the absolutevalue (|A−B|) of the level difference (A−B) to determine whether or notto add the edge component to the original signal as follows:

If |A−B| is greater than th, then the edge component is added to theoriginal image.

If |A−B| is less than or equal to th, then the edge component is notadded to the original image.

By performing the foregoing processing, it becomes unlikely that thesharpness effect is given to low-level noise components.

If th-0<th-1, setting th-1 further reduces the sharpness effect.

Moreover, to reduce the sharpness effect on the noise in the lowluminance areas of the image, a weight of negative gain is added to thegain Gs applied to the edge component of the luminance signal Y.

In order to enhance the appearance of the high luminance areas of theimage, a weight of a negative gain is added to the gain Gs to reduce thesharpness effect. These gain variations are shown in FIG. 11D.

The sharpness characteristics are also provided in advance in asharpness table 315 as optimized parameters and controlled according tothe parameter switch control signal 342 received from the parameterswitch control unit 341.

Lastly, the color signals Cr and Cb after the gamma processing aresubjected to saturation correction at a saturation correction unit 330.

In the saturation correction, gain is eventually applied to the colorsignal components to give a desired finish to the appearance of theimage.

As shown in FIG. 12A, according to saturation characteristics, lineargain is applied to the low saturation areas and low gain is applied tothe high saturation areas to minimize color saturation.

The saturation characteristics are also provided in advance in asaturation correction table 331 as optimized parameters and controlledaccording to the parameter switch control signal 342 received from theparameter switch control unit 341.

The switching of development parameters according to the firstembodiment will be described hereinafter.

The above-described development parameters are often changed dependinglargely on the amount of noise in the input image data. In other words,of the various shooting conditions, these parameters are selecteddepending largely on the sensitivity setting of the image sensor (i.e.,the ISO speed).

In multiple shooting, if the ISO speeds of the data of the images addedto generate a composite image at the composition processing unit 151 aredifferent from each other, it is desirable to use the developmentparameters optimized for the higher ISO speed with the larger amount ofnoise.

For example, when adding the first image data shot at ISO 200 and thesecond image data shot at ISO 1600 to generate a composite image, thedevelopment parameters are controlled as follows:

It is assumed that the 3×3 tap filter of FIG. 8A and the 5×5 tap filterof FIG. 8B are provided for ISO 200 and ISO 1600, respectively, in theluminance noise reduction table 308. In this case, the developmentcontrol unit 100 compares the ISO sensitivities of the first and secondimage data from the shooting condition information 117 and 164 andoutputs a development control signal 125 to select the developmentparameters for the higher ISO speed or the ISO 1600.

In the development processing unit 141, the parameter switch controlunit 341 outputs the parameter switch control signal 342 to a luminancenoise reduction table switch unit 309 according to the developmentcontrol signal 125. The luminance noise reduction table switch unit 309selects the 5×5 tap filter for ISO 1600 of FIG. 8B from the luminancenoise reduction table 308 and sends it to the luminance noise reductionunit 307.

In the same manner, it is assumed that the 5×5 tap filter of FIG. 8B andthe 7×7 tap filter of FIG. 8C are provided for ISO 200 and ISO 1600,respectively, in the color noise reduction table 322. In this case, the7×7 tap filter of FIG. 8C is sent out.

As shown in FIG. 9A, it is assumed that the color gain table 325 isprovided with characteristics 901 for ISO 200 and characteristics 902for ISO 1600, which apply a larger negative gain to the color componentsthan the characteristics 901. In this case, the color gain table switchunit 326 switches to the parameter table with the characteristics 902according to the parameter switch control signal 342 and sends it to thecolor gain unit 324.

It is also assumed that the characteristics 1001 for ISO 200 of FIG. 10Aand the characteristics 1002 for ISO 1600 of FIG. 10B are provided ineach of the luminance gamma table 312 and the color gamma table 328. Inthis case, the luminance gamma table switch unit 313 and the color gammatable switch unit 329 select the characteristics 1002 of FIG. 10Baccording to the parameter switch control signal 342 and sends it to theluminance gamma unit 311 and the color gamma unit 327.

At the sharpness processing unit 314, control is performed on the gainsGs and the thresholds th provided in the sharpness table 315. As shownin FIG. 11C, as for Gs (ISO200) and Gs (ISO1600) linearly applied toedge components, Gs (ISO200) is greater than Gs (ISO1600) to minimizethe amplification of noise components. The sharpness table switch unit316 selects Gs (ISO1600) according to the parameter switch controlsignal 342. In this embodiment, thresholds th (ISO200) and (ISO1600) areprovided, in which the thresholds th (ISO200) is less than the thresholdth (ISO1600). In this case, the threshold th (ISO1600) is selectedaccording to the parameter switch control signal 342.

In addition, in the weighting table for Gs shown in FIG. 11D, fromcharacteristics 1101 for ISO 200 and characteristics 1102, thecharacteristics 1102 for ISO 1600 is selected.

As shown in FIG. 12A, characteristics 1202 for ISO 1600 for minimizingincrease in color noise as well as characteristics 1201 for ISO 200 areprovided in the saturation correction table 331. The saturationcorrection table switch unit 332 selects the characteristics 1201 or1202 from the saturation correction table 331 according to the parameterswitch control signal 342.

Subsequently, the image data 317 after the foregoing sharpnessprocessing and the image data 333 after the foregoing saturationcorrection is outputted from the development processing output unit 334.

As described above, according to this embodiment, even if multipleshooting is performed under conditions with different ISO speeds, theuse of optimum development parameters ensures that appropriatedevelopment is carried out without increasing noise compared with singleimage shooting, thus providing a low-noise image.

Second Embodiment

A second embodiment will be described hereinafter with reference toFIGS. 4 and 5.

Note that, to focus the subsequent description on the features of thesecond embodiment that differ from the first embodiment, in FIG. 4,identical numerals/symbols are assigned to processes identical to thoseshown in FIG. 2, and, in FIG. 5, identical numerals/symbols are assignedto elements identical to those in FIG. 3 and description thereof isomitted.

In the development processing unit 141 shown in FIG. 5, a developmentcontrol signal 125 is inputted into the parameter switch control unit341 and a parameter composition control unit 543 from the control signalinput unit 340.

According to this embodiment, the development parameters for theluminance noise reduction unit 307, the color noise reduction unit 321,the luminance gamma unit 311, and the color gamma unit 327 are providedas discrete tables with respect to the ISO speed settings of the imagesensor 101. As in the first embodiment, the parameter tables areswitched at the table switch units 309, 313, 323, and 329 in response tothe parameter switch control signal 342 from the parameter switchcontrol unit 341 according to the ISO speed setting (S223 of FIG. 4).

On the other hand, the color gain unit 324, the sharpness processingunit 314, and the saturation correction unit 330, all provided withtables that can be linearly interpolated with respect to the ISO speed,compose development parameters according to the ISO speed (S427 in FIG.4) in response to a control signal 544 from the parameter compositioncontrol unit 543.

First, as for the color gain unit 324, a color gain table compositionunit 545 calculates parameter composition conditions from the shootingconditions of each shot of multiple shooting, composes a color gaintable 325 according to the calculated composition conditions, andgenerates new color gain characteristics.

Assuming that the characteristics 901 of FIG. 9B is for ISO 200 and thecharacteristics 902 is for ISO 1600, to add a shot at ISO 200 and a shotat ISO 1600 to generate a composite image, parameters are calculated toprovide characteristics having an intermediate gain between thecharacteristics 901 and the characteristics 902.

Furthermore, if an image is composed from two shots at ISO 200 and oneshot at ISO 1600, parameters are calculated to provide characteristics903 by dividing internally the characteristics 901 and thecharacteristics 902 in a ratio of 2:1.

In this way, a table of color gain characteristics composed by interiordivision processing in a predetermined ratio is provided to the colorgain unit 324.

As for the sharpness processing unit 314, a sharpness table compositionunit 546 calculates parameter composition conditions from the shootingconditions of each shot of multiple shooting, composes a sharpness table315 according to the calculated composition conditions, and generatesnew sharpness characteristics.

In addition, in the weighting table for Gs shown in FIG. 11E, thecharacteristics 1103 are calculated by dividing internally thecharacteristics 1101 for ISO 200 and the characteristics 1102 for ISO1600 in a ratio corresponding to the number of shots in the multipleshooting as in the foregoing color gain processing.

In the case of multiple shooting consisting of three shots (two shots atISO 200 and one shot at ISO 1600), gain Gs is calculated by interiordivision in a ratio of 2:1 as Gs=(Gs(ISO 200)×2+Gs(ISO 1600))/3.

Likewise, the threshold th is also calculated by interior division in aratio of 2:1 as th=(th(ISO 200)×2+th(ISO 1600))/3.

In this way, a table of sharpness characteristics composed by interiordivision processing in a predetermined ratio is provided to thesharpness processing unit 314.

As for the saturation correction unit 330, a saturation correction tablecomposition unit 547 calculates parameter composition conditions fromthe shooting conditions of each shot of multiple shooting, composes asaturation correction table 331 according to the calculated compositionconditions, and generates new saturation correction characteristics.

Assuming that the characteristics 1201 of FIG. 12B is for ISO 200 andthe characteristics 1202 is for ISO 1600, to add two shots at ISO 200and one shot at ISO 1600 to generate a composite image, parameters arecalculated that provides the characteristics 1203 obtained by dividinginternally the characteristics 1201 and the characteristics 1202 in aratio of 2:1.

A table of a saturation correction characteristics composed by theforegoing operations is provided to the saturation correction unit 330.

As described above, according to this embodiment, even if multipleshooting is performed under shooting conditions with different ISOsensitivities, new development parameters are generated by reaching aproper balance among the shooting conditions of the shots. Although thisincreases noise compared to the first embodiment, the second embodimentis capable of providing images reflecting optimum development parametersthat achieve a proper balance among the shots of multiple shooting.

Third Embodiment

A third embodiment will be described hereinafter with reference to FIG.6.

In the first embodiment, the parameter switch control unit 341 of FIG. 3automatically outputs a parameter switch control signal 342 according tothe ISO speed setting. In contrast, according to the third embodiment,the development control unit 100 performs parameter switching control inresponse to an instruction to switch the parameter table issued by theuser via the operation input unit 110.

Note that, to focus the subsequent description on the features of thethird embodiment that differ from the first and second embodiments, inFIG. 6, identical numerals/symbols are assigned to processes identicalto those shown in FIGS. 2 and 4 and description thereof is omitted.

Referring to FIG. 6, the processing at S201-S221 are identical withthose in FIGS. 2 and 4.

Subsequently, the development control unit 100 sets predetermineddefault development parameters (for example, for ISO 100) (S628), andthe development processing unit 141 performs development processingusing the development parameters set at S628 (S224).

Next, the CPU 112 displays the developed image data on the displaydevice 145 (S629).

The user then views the image displayed on the display device 145 anddetermines whether or not the image is properly developed. Ifdetermining that the image is inadequately developed, the user issues aninstruction for switching the parameter tables via the operation inputunit 110, and the development control unit 100 sets the developmentparameters for a different ISO speed (S628). As this embodiment allowsthe user to set the development parameters at the development processingunit 141, the user may perform fine adjustments to suit his preference.This, however, may result in too many parameters for the user to manage.As an alternative, for example, the entire parameters may be switchedfor each ISO speed or a different set of parameters may be provided inwhich parameter settings are assembled.

In this manner, once obtaining an optimum image after repeating theprocess of switching the development parameter setting and viewing thedisplayed image for each setting, the user issues an instruction forrecording the developed image via the operation input unit 110. Uponreceiving this recording instruction, the CPU 112 causes the encodingunit 143 to generate an image file from the developed composite imagedata 152 (S225) and records the file in the recording medium 108 (S226).

As described above, according to this embodiment, even if multipleshooting is performed under shooting conditions with different ISOspeeds, the user can view the displayed image and switch the developmentparameters via the operation input unit 110 if determining the image isinadequate. This enables fine adjustments of image quality to suit thepreference of the user.

The foregoing embodiments have been described with the ISO speed as theexemplary shooting conditions. However, even if other shootingconditions are used, the development parameters may be likewise switchedor composed if the development parameters require changes.

Furthermore, in addition to the shooting conditions, a resizing controlsignal and/or a brightness correction control signal 119 may be sent tothe development control unit 100 so that the degree of sharpness may beincreased with the resize rate, or a development control signal 125 maybe outputted to the development processing unit 141 so as to select atable for increasing the saturation correction when increasing thebrightness of the image by brightness correction.

Other Embodiments

Embodiments of the present invention can also be realized by a computerof a system or apparatus that reads out and executes computer executableinstructions recorded on a storage medium (e.g., non-transitorycomputer-readable storage medium) to perform the functions of one ormore of the above-described embodiment(s) of the present invention, andby a method performed by the computer of the system or apparatus by, forexample, reading out and executing the computer executable instructionsfrom the storage medium to perform the functions of one or more of theabove-described embodiment(s). The computer may comprise one or more ofa central processing unit (CPU), micro processing unit (MPU), or othercircuitry, and may include a network of separate computers or separatecomputer processors. The computer executable instructions may beprovided to the computer, for example, from a network or the storagemedium. The storage medium may include, for example, one or more of ahard disk, a random-access memory (RAM), a read only memory (ROM), astorage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2013-097999, filed May 7, 2013 which is hereby incorporated by referenceherein in its entirety.

What is claimed is:
 1. An image capturing apparatus comprising: an image capturing unit configured to capture images by an image sensor; an acquisition unit configured to acquire information relating to sensitivities of the image sensor used for capturing the images; a composition unit configured to add a plurality of images for generating a composite image; and a development processing unit configured to perform development processing of data of the composite image generated by the composition unit; wherein a plurality of parameters having different characteristics corresponding to the sensitivities of the image sensor are provided as the parameters used for the development processing; further comprising a control unit configured to perform controlling to compare the sensitivities used for shooting the plurality of images and set a parameter corresponding to the highest sensitivity of the plurality of parameters as the parameter used for the development processing.
 2. The apparatus according to claim 1, wherein the plurality of images added by the composition unit includes an image captured by the image capturing unit and an image decoded from an image file that has been already recorded.
 3. The apparatus according to claim 1, wherein the development processing includes at least one of luminance noise reduction processing, color noise reduction processing, luminance gamma processing, color gain processing, color gamma processing, sharpness processing, and saturation correction processing, a plurality of parameters are provided for each of the different types of development processing, and the development processing unit includes a parameter switch unit configured to select a predetermined parameter from the plurality of parameters.
 4. The apparatus according to claim 3, wherein the development processing is a nonlinear processing.
 5. A control method of an image capturing apparatus which has an image capturing unit for capturing images by an image sensor, an acquisition unit configured to acquire information relating to sensitivities of the image sensor used for capturing the images, a composition unit configured to add a plurality of images for generating a composite image, and a development processing unit configured to perform development processing of data of the composite image generated by the composition unit; the method comprising the steps of: providing a plurality of parameters having different characteristics corresponding to the sensitivities of the image sensor as the parameters used for the development processing, and performing controlling to compare the sensitivities used for shooting the plurality of images and set a parameter corresponding to the highest sensitivity of the plurality of parameters as the parameter used for the development processing.
 6. A non-transitory computer-readable storage medium storing a program for causing a computer to execute the control method according to claim
 5. 